Combining Multiple Representations for Pen-based Handwritten Digit Recognition

We investigate techniques to combine multiple representations of a handwritten digit to increase classification accuracy without significantly increasing system complexity or recognition time. In pen-based recognition, the input is the dynamic movement of the pentip over the pressure sensitive tablet. There is also the image formed as a result of this movement. On a real-world database of handwritten digits containing more than 11,000 handwritten digits, we notice that the two multi-layer perceptron (MLP) based classifiers using these representations make errors on different patterns implying that a suitable combination of the two would lead to higher accuracy. We implement and compare voting, mixture of experts, stacking and cascading. Combining the two MLP classifiers we indeed get higher accuracy because the two classifiers/representations fail on different patterns. We especially advocate multistage cascading scheme where the second costlier image-based classifier is employed only in a small percentage of cases.